A graph-based optimization algorithm for fragmented image reassembly

We propose a graph-based optimization framework for automatic 2D image fragment reas-sembly. First, we compute the potential matching between each pair of the image fragments based on their geometry and color. After that, a novel multi-piece matching algorithm is proposed to reassemble the overall image fragments. Finally, the reassembly result is refined by applying the graph optimization algorithm. We perform experiments to evaluate our algorithm on multiple torn real-world images, and demonstrate the robust-ness of this new assembly framework outperforms the existing algorithms in both reas-sembly accuracy (in handling accumulated pairwise matching error) and robustness (in handling small image fragments). Fragmented image reassembly recomposes a group of picture fragments into the original complete image. It is a geometric processing problem that has important applications in many fields such as archaeology and forensics. For example, archaeologists need to spend a lot of efforts to recompose fractured ancient paintings to restore their original appearances; forensic specialists manually compose damaged documents or pictures to recover the original evidence that are broken by humans. Developing robust geometric algorithms for automatic image reassembly can reduce the expensive human labors and improve the restoration efficiency, and hence is highly desirable. Existing automatic reassembly algorithms can be generally divided into two categories: color-based approaches and geometry-based approaches. Color-based methods mainly use the color information to predict the adjacency relationship of the fragments and guide the matching [1–3]. These algorithms are usually efficient, but they sometimes suffer from composition accuracy and may fail when the textures of different fragments are similar. Geometry-based methods reassemble the fragments by matching their boundary curves [4–7]. They can more accurately align adjacent fragments along their breaking regions, but they are sometimes slow and can easily get trapped in local optima then fail to reach the correct result. Therefore, incorporating both the geometry and color information in the reassembly computation could make process more efficient, accurate, and reliable. This paper presents a novel 3-step composition algorithm, as illustrated in Fig. 1. 1.1. Pairwise matching The first step is called the pairwise matching, which aims to identify adjacent pairs and compute their initial alignments. Geometry-based pairwise matching methods rely on analyzing the shape of the boundary curve contours ; color-based pairwise matching methods match fragments using their color information. Our integrated algorithm (1) extracts the border of each fragment and represents it as a curve contour, (2) clusters such a curve contour into multiple …